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Video encoding method and coiresponding computer progranme 



The present invention generaUy relates to the field of data compression and. 
more specifically, to a method of encoding a sequence of frames, composed of pictu« 
a^ments (pixels), by means of a three-dimensional (3D) subband dec^^^^^^ 
fi^^-;S^^^PaPPlied.inthesequenceconsideredasa3Dvolume.to«^^ 

w^chcorrespondinsaidsequencetoeachoneofsuccessivegroupsofframes(GOF^ 
GOFs bemg themselves subdivided into successive pairs of frames (POFs) including a so- 

^edpre^ous frame andaso-caUedcu^ent frame, said decomposition 
GOFs together with motion estimation and compensation steps performed in each GOF on 
saxds POFs and on corr^onding pairs of low-frequency temporal subbands (POSs) obtained 
10 at each temporal decomposition level. 

The invention also relates to a computer programme comprising a set of 

™ns for the implementation ofsdd«,codingmethod. when saidprogrammeisc^^ 
out by a processor included in an encoding device. 
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on..^ . "^"^^^^ (3D) subband analysis, based on a 3D. or 

(2D-.t). wavelet decomposition of a sequence of frames considered as a 3D volume has been 

jreandmorestudied for Video compression. Il^e wavelet transformgenerates 

that constitute a hierarchical pyramid in which the spatio-temporal relationship is defined 

tha^s to SDorientation trees evidencing theparent-of^ringdependencies between said 

coeffiaents. The in-depth scanningofthe generated coefficients in thehieraxehicalfr^ 

a progressive bitplane encoding technique then lead to a desired quality scalability" ~~ 

A practical solution for implementing this approach is to generate motion 
compensated temporal subbands usingasimpletwo taps wavelet fflter. as illu^^^ 

for a OOF of eight frames, in the iUustrated implementation, the input Video sequ^^^ 
dmded mto Groups of Frames (GOFs). and each GOF. itself subdivided into successive 

couplesofframes(thatareasmanyinputsforaso.caUedMotion-Comp»^^^^ 
F^ermg.orMCnTmodule).isfir^^ 

(TP). Haeresultmg low frequency (Dtemporal subbands of the first temporal decomposition 
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level are further filtered (TF). and the pmcess may stop after an arbitrary number of 
decompositions resulting in one or more low frequency subbands called root temporal 
subbands (in the iUuslration, a non-limitative example with two decomposition levels 
resultmg in two root subbands LL is presented), m the example of Fig. l. the frames of the 
5 xUustrated group are referenced Fl to F8. and the dotted arrows correspond to a high-pass 
temporal filtering, while the other ones correspond to a low-pass temporal filtering. Two 
stages of decomposition are shown (L and H = first stage ; LL and LH = second stage) At 
each temporal decomposition level of the illustrated group of 8 frames, a group of motion 
vector fields is generated (in the present example, MV4 at the first level and MVS at the 
10 second one). 

When a Haar multiresolution analysis is used for the tenq,oral decomposition, 
smce one motion vector field is generated between every two frames in the considered group 
of frames at each temporal decomposition level, the number of motion vector fields is equal 
to half the number of frames in the temporal subband, i.e. four at the first level of motion 
15 vector fields and two at the second one. Motion estimation (ME) and motion compensation 
(MC) are onlyperformed every two frames of the input sequence (generally in the forward 
way), due to the temporal down-sampling by two of the simple wavelet filter. Using these 
very sm>ple filters, each low frequency temporal subband (L) represents a temporal average 
of the mput couples of frames, whereas the high frequency one (H) contains the residual error 
20 after the MCTFstq). 

Unfortunately, the motion compensated temporal filtering may raise the 
problem of uncomrected pixels, which are not filtered at all (or also the problem of double- 
connected pixels, which are filtered twice). Tire number of uncom^ected pixels represents a 
weakness of a 3D subband codec approaches because it highly impacts the resulting picture 
quahty, particularly in occlusion regions. It is especiaUy true for high motion sequences or 
for final temporal decomposition levels, where the temporal correlation is not good. The 
number of these uncomrected pixels depends on the dense motion vector field that has been " 
generated by the motion estimation. 

Current criteria for optimal motion vector search used in motion estimators do 
0 not take into account the number of unconnected pixels that will be the result of motion 
compensation. Most sophisticated algorithms use a rate/distortion criterion which tends to 
nummize a cost function that depends on the displaced difference energy (distortion) and the 
number of bits spent to transmit the motion vector (rate). For example, the motion search 
returns the motion vector that minimizes: 
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Ih this expression (1). n. = K.-,f the motion vector, p = (^,.p,/ is the 
prediction forthe motion vector,and A„ is the Lagrange multiplier. Tlxe rate tenn 

R(m.p) represents the motion infonnation only and used as distortion measure is 

computed as : ' 

(note c«, b. ,). Unfo«y, ^ „„, 

r-s^my optafeadon, ^ a,^Ued to hybrid coding for wbioh *o invors. modon 
compensation is not performed. 



20 



25 



It is therefore an object of the invention to avoid such a drawback and to 
propose a video encoding method in which the set of uncomiected pixels is taken into 
account in the distortion measure. 

To this aid, invsnSon routes 10 a method iiuch a» defined in th« 
""-duaoty patagtsph and »hicl, i. naotoovor oh«™o,«i.«. in d>at. said process of motion 

comj^^edtentpora, Bering leadingintheprevionsta^s on me one hand to co.n«^^ 
pixels, that are filtered along a motion trajectory cotresponding to motion vector d,fin«i by 
mean, of satd motion eafimation «.ps. and », flte «her « to a tesidnal number of so- 

oailed^connected pixels, that arenotfitoed at all. .achmotion.stim.aon step compdsesa 
monon search provided for returning a motion vector m minimises a cost fhncfion 
depending at least on a distotsion criterion involving a distorfion measn,., said measnr. 
dtstoniion being also applied to the set of said uncomiKted pixels. 

The present invention will now be described, by way of «ampl.. with 

rrfer^c to ate accompanying drawing in which Fig. 1 showsat«nporalmultircsoludon 
analysis with motion compensation. 
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Because unconnected pixels highly participate to the quality degradation of the 
inverse motion compensated image, the set of unconnected pixels is. according to the 
mventaon, taken into account in the distortion measure. To this end. it is here proposed to 
introduce a new rate/distortion criterion that extends equation taking into account the 
) uncomiected pixels phenomenon. TOs is illustrated in equations (3) and (4), that are 
equivalent: 

K{m) = 7(m) + X^j^comEcrBD ' ^iSom^ (m)) ,3. 

with ^{S^coNNECTEDi^)) bcing the distortion measure for the set S: 
uncomiected pixels resulting from motion vector m . Several distortion measures can be 
apphed to the set of unconnected pixels. A very simple measure is preferably the count of 
unconnected pixels for the motion vector under study. 

It can be noted that the real set of uncomiected pixels resulting from a motion 
search can be computed only when the motion vectors infomiation is available for the whole 
frame. Therefore, an optimal solution can hardly be achievable (in fact a complex set of 
nmmnisation criteria for the whole frame should be solved), and a sub-optimal 
implementation is therefore proposed. This implementation, not recursive, can be considered 
as a smiple way to take into account the distortion due to uncomiected pixels. For a given part 
of the miage to be motion compensated (a part of the image can be a pixel, a block of pixels 
a macroblock of pixels or any region provided that the set of parts covers the whole image ' 
without any overlapping) and for a given motion vector candidate m . a temporary inverse 
motion compensation is applied, the set of micomiected pixels is identified, and 
J^iSuNco^n^im)) can be evaluated. The current K{m) value can then be computed and 
compared to the cmrent minimum value (m) to check if the candidate motion vector 
brings a lower Kim) value (for the first motion vector candidate, K (m) is obviously equal to 
the valeur JTCm) computed). When all the candidate have been tested, the (final) inverse " - 
motion compensation is applied to the best candidate (identifying comiected and uncomiected 
pixels). The next part of the image can then be processed, and so on up to a complete 
processing of the whole image. 

However, in this non-recursive implementation, the resulting decisions are not 
always spatially homogeneous over the whole image : for the first part of the image to be 
motion compensated, the set of uncomiected pixels may be empty, while the probabihty of 
uncomiected pixels for the last part of the image to be motion compensated is then very high 
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This situation can lead to heterogeneous spatial distorsions. In order to discard „ 
problem resulting ofthesing,e.passin.plen,entation,an.^^^^^^^ 
proposed, which indeed aUows to inxprove said single-pass one by minimizing the global 
cntenon ^ W for aU parts ofthe whole image, which can be done with a multiple-pass 
5 implementation including the following steps. 

First, for all the parts of the image, the optimal motion vector m„,. is 

computed, as weUasaset of sub-optimal motion vectors K^^jthat^^^^^^^ 

imnm^um values for y(m)ofequation(l). the number of unconnectedpixelsbeingnot used 
at th,sstage(thenumber of sub-optimal vectors N^^ is implementation dependent). For 
10 all these vectors, the corresponding value for the criterion J(m) is stored so that y(m., ) 

i^K-^-opj} are generated. Then an inverse motion compensation is applied for J 

optimal motion vectors m so that V jr/^ \ ^ 

»n.p, so tnat 2.K{m^ ) can be computed (note that TKim ) 

al I parts » I * opt / 

al t parts 

F»m 0„ U« Of ^boptaa. vecors, a,e c^didate motion vector 

I W™« )}- (4m„.™„ ))| is that selected (note that can b. a vector of any p.„ „f 

*e c™ i„ag„. For the «, of optitnal motion v«,to,a and U.e caadidat, v«,„r (in pUce of 

the „p„mal vector for the coneapondingpartofthe image), an invet^motioncomp^aation 
.a applt^, (") i. again comptrted. If its vatae is W tban Z'^(„^ ). Se 

al I parts 

optimal valueofm,, is replaced by m_,,, (for the corresponding part of the image). 
P^^y m..... '«^«<^^ded from the list of sub-optimal vectors. Ihenanew candidate is 
se]bcted and the same mechanism is appUed untU the list of sub-optimal vectors is empty in 
order to obtain the optimal set of motion vectors. - ' 



